Image refocusing may be used to refocus images after they are taken. For example, such refocusing techniques may be used to simulate a shallow depth of field after the fact (e.g., after the image or images are taken). A shallow depth of field in imaging may provide an aesthetic with a portion, region, or object-of-interest in an image being in focus and another portion or portions of the image being out of focus and having a blur. Such an aesthetic may be desirable to users. In typical photography, such shallow depth of field and related aesthetics may be attained using larger apertures and other photographic techniques while the image is being taken.
In image refocusing, there are several techniques to simulate a shallow depth of field. For example, plenoptic cameras may sample the light-field at the entrance pupil of the camera in 4D using a microlens array placed in front of the sensor that separates rays entering at different angles before they are averaged by the sensor. Such techniques allow refocusing after the fact by rendering views in which the integration to convert the 4D light-field to an image occurs at specific angles. The main disadvantage of such techniques is that the maximum aperture is limited by the physical aperture of the main lens and that it requires a complicated optical setup with a zoom lens in front of the camera. Furthermore, such zoom lenses are difficult to integrate in mobile platforms.
In another example, a camera array may be used to achieve refocusing after the fact. In camera array approaches, multiple images may be taken (e.g., via the camera array) and a shift and average technique may be used to achieve refocusing. For example, refocusing to a chosen depth includes selecting the displacement that corresponds to the depth, shifting the images (e.g., the multiple images obtained via the camera array), and taking the average of the images. Such techniques keep points that are at the chosen depth aligned and therefore in focus while points that are at other depths will not align and will appear blurred. Such techniques may thereby simulate a shallow depth of field as discussed. However, the results of such techniques may provide images that appear unnatural in the blurred (e.g., out of focus) regions.
As such, existing techniques do not provide refocusing after the fact with ease of use and high quality. Such problems may become critical as the desire to obtain high quality, aesthetically pleasing images in a variety of device implementations becomes more widespread.